The present invention relates to a computer-based system and method for testing computer software (code).
Distance learning is an educational process in which a student and an instructor are separated from each other in time, space, or both.
A distance learning technology is one that facilitates distance learning. Examples include the Internet and interactive video.
Although there have been some long-standing distance learning institutions (e.g., National Technical University, U.S., and Open University, U.K.), there has recently been an explosion of interest in distance learning on the part of universities. The driving technology behind this trend has been the Internet, or the world-wide web. Distance learning depends on the web""s ability deliver instructional material to students. Distance learning also benefits in many fields from the web""s ability to interact with students. In some disciplines, for example, literature, web-based interactivity is not viable. In others, especially quantitative disciplines such as chemistry or physics, web-based interactivity is quite workable. Web-based interactivity increases educational effectiveness and decreases cost.
Ironically, web-based interactivity in most areas of Computer Science has not, in the past, been achievable. This is because the xe2x80x9cproductxe2x80x9d of a Computer Science exercise is a fragment of a program (e.g., a statement, statements, or portion of a statement which does not, in and of itself, constitute a complete computer program), or occasionally, a complete program. Because there are many ways of writing a correct piece of code, and even more ways of writing an incorrect one, multiple-choice, textual fill-in, true/false and other standard modes of interactivity used in other fields are inappropriate for Computer Science instruction.
The present invention solves this problem by allowing web-based distance learning software to interact with students writing code fragments.
The shortage of computing professionals in the United States is well known, and has led to calls by leaders in the computing industry for special immigration dispensations. Yet, this shortage is not without great irony. Each year, great numbers of students embark on a Computer Science major, only to be lost through retention failure. It is common for fifty per cent of the cohort to fall by the wayside through each of the first two or three courses in the undergraduate Computer Science sequence.
The cause of this phenomenal attrition rate lies in one of the remarkable differences between math education and foreign language study, on the one hand, and Computer Science education, on the other hand. Whereas language and math students are given large numbers of drill exercises, Computer Science homework tends to consist of a small set of assignments in which complete computer programs must be written.
Writing a complete program requires simultaneous integration of many cognitive elements: syntax and semantics of language constructs, elementary use patters, algorithms, and so on. Without mastery of these elements, students cannot make headway with assignments. Frustration ensues, and the point of the homework is lost. Some students managexe2x80x94either through intuition, prior experience, or perhaps personality idiosyncrasiesxe2x80x94to carry out these assignments. Many fail, however, due to the lack of the opportunity to master the nuts and bolts of programming.
One solution to this problem is to reform Computer Science pedagogy by introducing programming drills, not to replace traditional closed labs and programming homework, but to provide a basis for student success in those areas.
It is remarkable that a technique that is fundamental in the teaching of foreign languages and mathematicsxe2x80x94both closely related to Computer Sciencexe2x80x94is unused in Computer Science education today. A question to ask is: if drill is so useful, why isn""t it standard in Computer Science education?
The answer lies in the nature of computer code. A response to a short exercise in a mathematics or foreign language can be easily and reliably checked by a human being. Students themselves can check their own solutions against a given set of answers. This is not the case with programs or even modest program fragments. Code can be verified manually only with a great expenditure of human effort. Without a means to check answers, drill is meaningless.
The present invention solves this problem by facilitating the creation of programming drills, and providing automatic, reliable checking of such exercises.
Nearly all distance learning technology used in Computer Science education is implicitly modeled on or explicitly imported from non-Computer Science disciplines. These systems are characterized by delivery of material (textual, graphic, animation), on-line real-time or asynchronous discussion groups, xe2x80x9cinteractivexe2x80x9d questions of the traditional multiple-choice/textual fill-in type with instantaneous feedback, on-line tests based on questions of this type, collections of homework and roster maintenance.
Distinct from the above described comprehensive distance learning systems are automatic homework program checking systems. These systems check the correctness of entire programs or at least complete modules thereof, giving instant feedback to the student. They have been around since the mid-1980s and reflect the naturally evolving automation of steps that an instructor would take to check the correctness of a submitted homework program by trying to run the program directly on a computer and witness the results. These systems are characterized by the following limitations:
correctness is determined by comparing output of a single file with an expected output;
an assignment must be a complete program or program module; and
the system works with only one programming language.
Finally, there are some cases of systems that check other characteristics besides correctness, e.g., style or various software quality metrics. These systems suffer from the last two of these above-listed limitations.
The present invention is directed to a system for checking the correctness of computer code fragments. In an example embodiment, a web-based automatic code correctness checking tool is provided for distance learning. A student may submit to the system a solution to a selected exercise. The solution is processed and verified by a server system, which may provide hints and/or error messages to the student if the solution in incorrect. The present invention is also directed to a system and method for setting up exercises for distance learning courses.